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ABSTRACT 



T!:e main purposes of this investigation were to determine 
whether statistical sampling- techniques could be applied ef- 
fectively in follow-up studies of high school graduates and, 
if so, to determine what sampling model is most appropriate 
for that purpose. ' ' ■ ■ 

The 1970 and 1971 graduates of four high schools in the 
Santa Barbara City area served as the populations of interest. 
The high school. .re cord of each graduate was analyzed and the 
following information was recorded: the graduate's name, sex, 
address, phone number, parent's name, ethnic background, grade 
point average, graduating class rank, Lorge-Thorndike Verbal 
Test score, citizenship grades, years at the school, extra- 
curricular activities, and the types and number of courses of 
each type he had taken. These data were used to develop 
statistical, demographic profiles of the two populations. 

A questionnaire that had been specifically developed to 
provide actionable data and' that had been pretested to evalu- 
ate its comprehensibility and conciseness was sent to each 
graduate. Three weeks later a second questionnaire was sent 
to the non-respondents and three weeks after that the non- 
respondents were interviewed by telephone. Data from 78 per- 
cent of the 1970 and 82 percent of the 1971 graduates were 
thus obtained. 

The statistical profiles of the respondents with respect 
to the previously mentioned variables recorded 'from their stu- 
dent record cards were very similar to the population profiles 
so that, in essence, the questionnaire data were representative 
of the entire population. However, differences in the response 
rates and in the responses to the questionnaire items were 
found among the respondents. These differences were analyzed 
and found to bo related to the sex, ethnic background, and 
grade point average of the respondents. 
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Subsequently, different size random and stratified-rando» 
samples were drawn from the respondent pools of the two popu- 
lations. The latter type samples were stratified on various 
combinations of sex. ethnic background, and. grade point average. 
Several behavioral and attitudinal items appearing in the ques- 
tionnaire were selected to evaluate the effectiveness of the 
sampling models. When the simple statistics were compared to 
the population values, it was found that the stratif ied-random 
sample models provided more accurate results than the random 
sample models.*,,,- 

It was concluded that statistical sampling techniques 
applxed to follow-up studies of high school graduates could 
provide accurate data in a cost-ef f ective manner and it was ^ 
recommended that samples stratified on sex, ethnic background, 
grade point average, and high school program be used in such 
studies in the future, as should the data collection procedures 
developed in the present investigation. In addition, several 
criteria for selecting the proper size sample were discussed 
and recommendations made based upon those criteria. 
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PROJECT TRACE i 
RESEARCH AND EVALUATION OF FOLLOW-UP STUDIES 
OF HIGH SCHOOL GRADUATES USING SA.MPLING TECHNIQUES 



INTRODUCTION 

There currently exists a need to develop an effective 
and economic means of obtaining up-ta-date evaluative data 
on recently implemented educational programs in the State of 
California, particularly at the secondary school level* Per- 
haps, to a greater extent than ever before in our history, 
significant changes in educational philosophy, in curricula, 
in communication media, in instructional emphasis , and in 
policies and practices have been instituted in the schools of 
our nation* In coming years, social and educational pressures 
upon the educational system will undoubtedly continue, with 
the probable result that more changes and innovations will be 
made. Some of these may possibly result in improved educa- 
tional programs; others will not. 

One method used in the past to obtain evaluative data 
has been the follow-up of recent graduates. In conducting, 
follow-up studies, it has been common practice to attempt to 
Cj)llect information from the entire graduate population 
rather than from representative samples of the population. 
This is both impractical and unnecessarily expensive. 

The major purpose of the present investigation was to de- 
termine whether statistical sampling techniques commonly used 
by survey research organizations could be applied effectively 
in conducting follow-up studies of high school graduates and, 
if so, to determine what sampling model is most cost-effective. 
A secondary purpose was to develop a concise, self-administered 
questionnaire which could provide aationable information from 
graduates, that is, information which could be put to direct 
use in the development of effective programs for future students. 



The Problem 



The use of follow-up studies to evaluate educational 
programs, however, is not, without problems. An issue of 
fundamental concern is how" often and how long after gradua*- 
tion attempts should be made to obtain the evaluative data. 
Is it satisfactory, for example, to -contact the graduates 
only once, perhaps a year after graduation, or should the in- 
formation be obtained periodically over a period of several 
years after .graduation? The latter approach seems more de- 
fensible in that it is unlikely that the full consequences 
of a particular program would be manifest shortly after grad- 
uation and, further, the consequences very likely would 
change over time. A reasonable approach might be to follow 
up a graduating class for a period of from, say, four to five 
years at intervals of perhaps a year between follow-ups. If 
this approach were used in follow-up studies of four consecu- 
tive graduating classes of 600 students each, and if, as has 
been the custom, an attempt were made to obtain information 
from the entire population, then in the fourth year a total 
of 2,400 potential respondents would have to be contacted and 
their data processed and evaluated. The amount of administra- 
tive manpower needed for such a project would be great, as 
would be the associated costs; generally, they can be expected 
to be beyond the scope and budget of individual schools and 
school districts. 

Another problem that often arises in follow-»up studies 
stems from the typical response rates. The rates for various 
follow-up studies have varied from about 45 percent to 97 per- 
cent, more often falling at the lower end of this range. 
Thus, the validity of the conclusions that can bo drawn from 
the obtained data may be suspect. High response rates (90 
percent and above) appear to be obtainable only when inten- 
sive orientation programs for the students have been previously 



carried out. But of greater importance than the absolute 
response rate is the representativeness , by the respondents, 
of the population. Most often this is a problem when response 
rates are low and various segments of the population respond 
disproportionately. For example, if graduates who are scho- 
lastically below average or who are members of minority groups 
do not respond as readily (Rs^is oft^en the case) as other 
graduates, the resultant data*^inay not accurately reflect the 
experience and attitudes of the entire graduate population. 
Though this pi;oblem might be partially resolved using statis- 
txcal weighting techniques, a great deal of the data obtained 
from those groups of students who are proportionately over- 
represented among the respondents are unnecessary for evalua- 
tive purposes. Thus, there results a waste of time and money 
in obtaining, processing, and analyzing all the follow-up 
information of the overrepresented groitns. 

Alternative Approaches 

Both of these major problems--the required manpower and 
associated costs of administration, and the difficulty of ob- 
taining representative, valid data--are due for the most part 
to the attempts made to gather information from the entire 
graduate population. An alternative approach which may alle- 
viate these problems is to apply statistical sampling tech- 
niques in follow-up studies. With such an approach there is 
no need to obtain information from the entire population; 
rather, data are obtained from a number of carefully selected, 
representative graduates and inferences are then drawn from 
their responses and projected to the entire population. Be- 
cause only a relatively small number of graduates need be con- 
tacted, the administrative and cost requirements are reduced 
and, to the extent that the selected graduates are representa- 
tive of the population^ valid inferences can be made to the 
entire population and valid conclusions can be reached. 
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There are various sampling techniques which may be used* 
Associated with each, however, are advantages and disadvan* 
tages. Two of the more promising techniques for use in follow- 
up studies are (1) the random sample technique, and (2) the 
Btratified-random sample technique • 

The most easily selected sample is the random sample* In 
such a sample, each individual* in tKc population of interest 
(in this instance, the graduates of a given year) has an equal 
chance of being, selected. While this sampling procedure is 
relatively inexpensive and'easily implemented, it does have 
some drawbacks. Because the sample is selected randomly, it 
is difficult to ensure that the diverse segments of the popula- 
tion, e.g., good and poor students, minority and non-minority 
group members, etc., are proportionately represented. Though 
this problem can be remedied somewhat by increasing the sample 
size, thus making it more likely that all segments are repre- 
sented, that solution also serves to increase the manpower and 
cost requirements . 

A strati fied^random sample differs from a random sample 
in that the population is divided into subgroups or strata 
before the sample is drawn. The subgroups are identified by 
variables thought to be related to the outcome of the study 
(e.g., ethnic group or grade point average). After the strat- 
ification is accomplished, a random sample is drawn from each 
stratum. Usually the size oH the sample from each stratum is 
proportional to the size of the subgroup in the population; 
however, disproportionate samples may be drawn from the strata 
depending upon the particular aims of the study. The strati- 
fied sampling procedure serves to ensure that various segments 
of the population arc properly represented in the sample and 
thus increases the likelihood that valid inferences may be 
made to the population. 

The main disadvantage of stratification is its associated 
costt The population members must be classified according to 
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the variables of interest and, obviously, the more variables 
and the more strata used, the more laborious and costly the 
procedure. Thus, a major consideration in implementing a 
stratified-random sampling approach is the selection of crit- 
ical variables and the number of strata to use within each 
variable^ 

Another consideration in,, inplemejating sampling techniques 
is the size of the sample to be drawn'" from the population. The 
larger the sample, whether random or stratified, the greater 
the confidence -one can have that the conclusions drawn from it 
can bo generalized to the population. However, mere size does 
not assure representativeness in a sample. The more diverse 
the population (with respect to some critical variabl e [s] ) , 
the less likely that the results obtained from two or more in- 
dependently drawn samples would be the same. Thus, the data 
from a small random^ ojr s,tratified-rando'm sample is likely to 
be superior to a larger but badly selected sample. 

The present investigation was designed to evaluate alter- 
native sampling plans that can be used in conducting follow-up 
studies of high school graduates* The elements of the study 
included assessments of the sampling plans, the sample sizes, 
the information gathering and measurement techniques, the data 
collection methods, and the cost-effectiveness of the various 
alternatives for each of the elements. 



Method 



The study was conducted in three major phases and with 
two populations, the 1970 and 1971 high school graduating 
classes in the Santa Barbara City area* The participating 
schools included Santa Barbara High School/ San Marcos High 
School, Dos Pueblos High School", and-G-arpint eria High School.^ 
In the first phase, the high school records of all the gradu- 
ates were analyze.d and a follow-up questionna . developed 
In the second phase , an attempt was made to obtain fol low-up 
data from all the graduates; and in the third phase, various 
sampling methods were evaluated using the data obtained in 
Phase 2. 

The general procedure was to categorize each of the grad- 
uates on certain classification factors and to obtain behav- 
ioral ard attitudinal data from them through suitably prepared 
questionnaires* The resulting data were evaluated with re- 
spect to the classification factors and statistical descrip- 
tions of the populations were formulated. Differences in 
attitudinal and behavioral data and in response rates, as 
functions of the classi/ication factors, served as the basis 
for the selection of stratification variables which were used 
in samples of various sizes. The samples were evaluated by 
comparing the sample values with the population values on 
several of the attitudinal and behavioral questions. The pro- 
cedures used and the relationships among procedures are shown 
in Figure 1* They are described in detail in the following 
sections . 



^Another high school, La Cuesta, also participated in the 
initial two phases of the study* However, owing to the 
small number of graduates from that school and the fact 
that La Cuesta is a continuation school rather than a 
typical high school, the data obtained from its graduates 
were not used in the third phase of the study. 
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Procedures 



t^hase 1 - Informational Requirements^ Record Analysis^ and 
Questionnaire Development 

In this phase of the study, the objectives were to gain 
the cooperation and to learn the informational needs of the 
local high school administrators and *t:ounse lors , to conduct 
the record analyses of the graduates, and to develop a ques- 
tionnaire that would provide actionable data, 

.Meetings were held with the project steering committee 
(representatives of the Santa Barbara County Schools) and 
high school principals, administrators, counselors, and other 
interested members of the educational community. The initial 
meetings, some of which were held with individual school per- 
sonnel and some with all the schools represented jointly, were 
devoted to descriptions of the project objectives and its po- 
tential usefulness to these personnel. Having thus obtained 
their interest and willingness to participate, it was then 
determined what soit of information they^ desired from their 
graduates and which variables they thought would be pertinent 
in classifying them into subgroups for stratified sampling. 

Lists of potential items for inclusion in the question- 
naires were devised, based upon the results of the meeting. 
These were forwarded to the meeting attendees for study, eval- 
uation, and comment* The attendees were also asked to formu- 
late potential questionnaire items which would be considered 
and discussed at a subsequent meeting. 

At the same time, sample student records were obtained 
from each of the schools and examined as the cl assi fication 
variables to be used would be determined by the contents of 
the records. The purpose of this step was not only to deter- 
mine the record contents but also to assess the commonalities 
and differences among the record-keeping practices of the 
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schools. It was found that although much the same information 
was recorded by each of the schools, the techniques of record- 
ing differed significantly. For instance, citizenship grades 
in some schools were recorded as numeric values, but in others 
they were coded in letter form. Some schools had photographs 
of the student attached to the record while other schools did 
not. And some schools maintained listings of the extracurric- 
ular activities the student participated in while others did 
not have such listings on the record. In addition, the loca- 
tion of specific. information on the record varied from school 
to school. ^It therefore became necessary to develop different 
techniques for analyzing the records of the various schools. 
In some instances it was also necessary to rely on other docu- 
ments and records, such as yearbooks and registration forms, 
for various kinds of information. 

At the subsequent joint meeting of the steering committer 
and high school personnel, decisions were reached regarding 
the items to be included in the questionnaire and the manner 
in which the student records were to be analyzed. 

A preliminary form of the questionnaire was designed and 
pretested on a group of 1969 graduates from Santa Barbara High 
School. The participants, 8 males and 5 females, represented 
several ethnic and socioeconomic groups and varied educational 
objectives. After each person completed the questionnaire, he 
was interviewed to determine what difficulties he encountered 
with either its wording or format. A few items were modified 
as a result of the comments offered though, in general, the 
graduates felt that the questionnaire was well-worded and 
easily understood. A final version of the questionnaire was 
then designed for field use. It is shown as Figure 2.^ 

^Analysis of the responses received from the 1970 graduates 
suggested that two of the qucs tionnviire items were not easily 
understood by sone members of that class. Therefore, the in- 
structions were altered slightly for field use with the class 
of 1971. It is the latter version which is shown as Figure 2 
The modified instructions were for Items 17 and 23. 
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PROJECT TRACE QUESTIONNAIRE 



If the information on the label at the left is incorrect, please enter the correct 
information at the richt. 



NAME 



last 



first 



middle 



ADDRESS 



no. 



street 



ci ty 



PHONE 



state 



zip 



area code nunber 



Please follow instructions carefully and answer ail questions 
that apply :o you. 



1. What are you doinq at the present time? (Chefk each Item that describes your 
^ - . - , J ..... 



till present activity, 



Oin school, full-time 
OIn schooU part-time 

OIn armed forc^s^ .v.*^ ' 
<• OHousewl fe > ' 
«-O0ther (please describe) 



f- OWorki ng , f ull - ti ne 
<-OWorki ng , part-time ^. 

i-ONot v/orking, looking for a job 
••ONot working, not looking for a job 



Complete the questions i.i thir box if you are attending school FULL-TIME or PART-TIMt 



2. What type of school are you attending? 

'•OFour-year college or university 

«*0Technica1 school (type) 

5'OTrade school {type} 
\ <-OPrivate business school 
t 

•-OOther (describe) 



»-OJunior college (also check one belov/) 
_{tji i.QPlan transfer to 4-year college 
" »-OTwo-year AA program only 

j-OCertificate program (describe) 

<'O0ther (describe) 



1. Vlhat is the name of the school? 
What IS your major subject? 



ONone 



5. Compare the average grades you received in high school v/ith the average grades you 
are receiving in your present school by placing a single mark on each line scale. 



High school F 
grades^ J»> L 

I- 

Present school F 
grades i»>L_ 



s* 
D 
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A 



Conplete the questions in this box if vou arc working FULL-TIMH or PART-TIME or if vou 
arc LOOKING FOR A JOB, 



6. Which of the follov/ing is your present job situation? 
(It) 

i-OUncmployod , but looking for a job 
>-Oln an apprent i cesh i o prooram 
j-ORcceiving on-the-job training 
*'Olrt a job I am fully qualified for 



7. How many months of part-time employment have you had since finishing high school? 
mos. 



How many months of full-time onpl oynent?' r?os . 

How many months have you worked at your prJTeiit job? 



mos 



III. i*i 




rigurc 2. The questionnaire. 
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8. What business or industry are you employed in? 

9. What is your job title? 

. 



10^. Which siiqle_ statement best des-cribes your job? 

I li'M ^"'l" '^^l'!' ^ '•e^r'^ed specific high school training 
•O n f tl'li '^^'"^^^ i° ^'^^ training ^ 

' OIn a field unrelated to my high school training 

11. Indicate your weekly salary by placing a marTon the line scale. 

50 $50 $100 $150 ' 5200 



u»» 



$150 

-_L_ 



$250 or more 

J 



DISAGREE 




«0T SU^£/ 

•;v2£ci:-i:. 


=5111 


STKOSuLY 


1. 

(tei 0— - 




— -0— 


4. 

-0- 




CJil 0 


o 


0 


o 


o 






















C«l 0 


o 


0 


o 


o 













chances for advancement are good* 

The pay is good, considering my training 
and experience* 

I am satisfied with this job, at this stage 
in my career* 

The job is related to ny ultimate occupational 
objective. 



Please answer all of the following questions 



13 



. What is your occupational choice now? ^^^^^ 
OHave not rnade a choice. ^ 

14. What was your occupational choice when you were in high school? 
«• OHad not .rade a choice. 



07. }i) 



15. 

I* 

. f- 
3- 



What educational program did you take in high school? (Check one.) 



pCcllece Dreparatory 
OVocat1c-2l progran {which one?) 
OGeneral program 
OOther (cescribe) 



16. 

I- 



What educational program would you take now if you had it to do over again? 



OSame prDcran 
OA differe.ot program: 

»-OCol 1 ege preparatory 

'•0 Vocational proqram {which one^) 

I'OGone'-al program 

••OOther (describe) 



ERLC 
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17- 



Read the foUov/ing list 
column according to th^ 



of high school course areas* 
instruction at the toR.: 



Then make checks in each 



Did you take 
any courses 
in this area? 



If "yes/ check the 
degree of usefulness 
of courses you have 
taken to your pre- 
sent activl ty 



Course areas 



Agriculture 

Art 

Business education • 

Drama 

Distributive education-— 
English 

Foreign language ^ — 

Homemaking & consumer ed 

Industrial arts .-.-.w 

Hathematics \ • 

Music 

Science 

Social studies — 

Speech 

Vocational education 
(identify) 



no yes 

•<««iO--0 o--- 



not sonewhat 
useful useful 
-—7^ 



1-1 



<«o o 

— ut,o— o 

<*»iO o 

<«o O 

-.:«:C>-0 

<«o o 

—i««?0-0 



o 



very 
useful 

o 



Check here if 
you wish you had 
taken any OR 
more courses 
in this area 



-rt:_0- O O 



««)0 o 



— 0-— 
0 

— 

O 

-O— - 

.0 

— o— ■ 



-~!J,;0— 3 



-8 -8-- 



O O 

— o o— 

o o 

— o o— 

o o 

---o o— 

o o 

— o o— 

o 

■o— 
o 



Work experience 
(type of work) 



(M, fi-.O-O 

uj. itiO Q 

w. ** (y=KJ 



-0 O O". 

o o 
•-o -o 



— o 
o 

— o 
o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

-O 



o 



18. In your present activity (school, work, home, etc.), which course area, listed 
In Question 17 , is 



(M. Ml Most useful 

•"•«•! Second most useful 
Uf. Ml Third most useful 



Please write 
only one 
course area 
in each space 



19. Are there any specific courses that would be useful to you in your present 
<»» activity, but were not oiierea in your high school? 



••OVes (please describe) 
»0«o 



(M. ») 



20. If you did not take a ',/ork Experience program, v*hich of the following was you- 
«'•» reason? 



' Oflot interested 

»-ONo program was available 

'■ONo academic credit was offered for work experience 
*-pi could not ge; ..crk experience in my area of interest 
•■OOthcr (please describe) 



21. Who helped you rest in planning for the future when you were in high school? 
'«» (Check one.) 



'•QParents and relatives 
«-O0n-caiiipus frienos 
'•QOf f-campus frioncs 

••OOthers (Please describe) 



'•OSchool counselors 
»• OTcachcrs 

«-OWork experience advisor 



10b 



22. In your opinion, how much did high school help you gain the following everyday 
living skills? 



Mot Somewhat Very 
Helpful Helpful Helpful 
I* I- }. 

O— -----O — O Reading for my own instruction and pleasure 

O O O Writing letters, reports, or notes accurately 

luiO-"- — O --*-0 Working out numerical problems and managing finances 

<«*>0 O O Speaking before qrouos of people 

|ii;0 O -^-^O— P»^e'paring for marriage and family life 

'•''Q O O Getting and maintaining a job 
(;;iQ ---O O Using proper spoken English 

08;O O O Tiiinking out problems and issues and forming my 

own opinions 

23. Read the following statements and -rank them -in the order of their importance to 
you in making an occupational choice ^ For examole, if you feel to "become 
well-known" Is nost ir.porcanc, write a '^1" next to that statement. Write a *'2" 
next to the second most icDortant statement, a "3" next to the third most im- 
portant, and sc on, up to 9, until eaca statement is ranked. (Use each number 
only once. ) \ ^ ^ 
% " * 

{III ^ Have a secure job 

(ttir I Become well-known 

Have a large i ncome 



itu 

fn» Be able to pursue my own interests 

cm . Have a low-pressure job 

o*) Have a job that is respected 

cm ^ Be able to help others 

Be able to express myself through my work 

•.2^ Be in charge of people ^ 

24, Read each statement carefully and make a check in the circle that best describes 
your agreement or disagreement witn the statement, 

Dis/G'ai §l$A22iI Aa^.;; , 

Q Q Q X X . "y counselor gave me good advice in planning my 

J V -v^ V/— ^^g^ school program, 

cjoO n n n r\ counselor gave me good advice in plannino my 

^ ^ ^ ^ ^ activity after graduation, 

miQ O O ^^^^^ ^ 9°°^ ^^^^ useful discussion between 

' students and teachers in my high school. 

(jj)0 n n C) n teachers tried to present information 

j ^ \j ^^ \^ Kj relevant to .today's world. 

» <j«>Q o O- O- -C>- '^'^^^ ^y teachers were not prejudiced toward 

minority group members. 

(»:0 O O O O of teachers did not show favoritism 

toward college-bound students. 

CM>0 -O O O O*- ^ ^^^^ ^^^^ ^''^^ school courses were very useful 

!in helping me meet my career goals. 
iviQ O O O O ^ ^^^^ ^^^^ ^^^^ school was a pleasant and reward- 

ing cxperi cnce . 

{»',0 0 O O— A person who works hard can usually get ahead. 

j ^«»0 0 O O O Succeeding is largely a matter of being in the 

* ^ right place at the right time. 

c^ojQ 0 O O O- People who get ahead usually have connections. 

C4oO O O O O ^^^^ society tends to favor people who have a 

' ^ different outlook on life than I do. 

cijiO n n O n ^ don^t think I would be very interested in most 

* ^ ^ ^Ijg ^1^^^ J qualify for. 

c«,0 O O O O chances for success ore probably better in sor/.e 

^ foreign countries such as Canada or Australia. 

' O Thank sou tor compbfim -Ihe qucsf lonnciirc. 

ERLC ^ ^ ^ J L 
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A data sheet was designed for use in the student-record 
analysis task* The format of the data sheet (Figure 3) was 
such that it could be used at all of the schools with a mini- 
mum of effort and chance, for error. With this form, informa- 
tion was obtained about the school the graduate attended, his 
year of graduation (and whether he graduated mid-year), the 
graduate's name, student number, se-x, address, phone number-, 
parent's name, and ethnic background. The parent's name was 
recorded to permit the tracing of the student, through the 
phone directory, in the event he had moved. The ethnic back- 
ground of the graduate was determined by examis^ing his photo-- 
graph, his surname, and in some instances by speaking with 
his counselor. Other information recorded on the data sheet 
included~th'e graduate's rank in his class, his citizenship 
grades, his Lorge-Thorndike Test scores, whether he had taken 
the college entrance examination CCEEB) , his overall grade 
point average, and the number and types of extracurricular 
activities he participated in. Finally, the analysts re- 
corded the number of semester courses the graduate had taken 
in a variety of subject areas listed at the bottom of the 
sheet . 

The record-analysts were a group of carefully selected 
college graduates; most were socif.l science majors who had 
had experience in data and record analysis of some sort. 
They were informed about the purpose of the study and were 
enthusiastic about participating in it. After they had been 
trained, they performed the analyses at their assigned schools 
where they quickly established rapport with the counselors and 
registrars, who in turn provided assistance whenever it was 
needed. 

The information on the completed data sheets was then 
coded and keypunched onto computer cards. Thus, data on each 
of the 1970 and 1971 graduates was obtained and readied for 
computer processing. The data from each graduating class were 
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PROJECT TRACE RECORD ANALYSIS 



) Carpinteria 

] Dos Fueblos 

] La Cuesca 

] Santa Barbara 

] San Marcos 



(high 
school) 



C ) 1970 
I ) 1971 



{ ] mid-year 



(graduation 
year) 



Last 



First 



Middle 



cr 



f ] Male 
i ) Female 



(sex) 



Number 



Street 



City 



Zip 
Grad« Phone 



Parentis Name 



] Caucasian 

) Spanish Surname 

) Black 

] Oriental 

) Other 

] Unknown 



10 
11 
12 



I J 

I J 

[ ] 

[ I 

[ 1 

[ ] 

[ ] 

[ ] 



_ of 

1 2 



(Class Rank) 



4.0 
3.0 
2«0 
1.0 



X, Verbal 

(Lorge-Thorndikc) 
_% Non-Verbal 



I ] 

I 3 1 

I ) 2 

I J 3 

[ ] 4 



C££B 



(Check if taken) 



(Years 

at 
School) 



(Overall CPA) 



student governnent 

athletics 

school clubs 

school publications 

pep club 

cheerleader 

others 



no information available 



/ 



completed by 



date 



_ Art 

Business 

Dance 

Drafting 

Drama 
. Engl ish 

Foreign Language 

Homcmaking ^ Consumer 

Industrial Arts 

Hath 

Music 

Science 

Social Studies 
Speech 

Work Experience 

Other _ 

Other " 

Other ^ 



Agriculture 
. Construction Trades 
Data Processing/Computer Programming 
Distributive Education 
Electronics 
Floristry 
Law Enforcement 
Motor Vehicle Repairs 
Other 

Other " 

Other ' ^ 



Nursery School Aide 
Nurse's Aide 

Service Station Attendant 



Figure 3. Student-record analysis data sheet. 
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anal/zed separately and" population profiles were obtained. 
These profiles described the percentage of graduates in each 
-of— the subdivisions of the classification factors derived from 
the student-record analyses. 

Phase 2 - Data Collection 

The behavioral and attitudinal data from the two popula- 
tions, the classes of 1970 and'l971 , li'ere collected consecu- 
tively. For the 1970 class, data collection began in December, 
1971; for the 1971 class, data collection began in April, 1972. 
The data were' collected in this manner so that the techniques 
used for the 1970 class could be evaluated and, if necessary, 
modified for the class of 1971. As was mentioned earlier, the 
questionnaire was modified slightly for field use with the ,1971 
class. The data collection procedures used for both classes 
were, however, virtually identical with the exception of some 
minor details which will be described* 

The questionnaires were professionally printed as were 

mailing and return envelopes. A first-class mailing permit was 

purchased and printed onto the mailing envelopes; the envelopes 

also had a printed return address in the upper left corner.* 

The return envelope was prepared in a business reply format 

with the same address as above. (For the class of 1971, both 

to 

the mailing and return envelopes were prepared with conventional 
postage stamps, though the project address was printed as before 
on both envelopes. It was felt that this approach would lend a 
more "personal touch" and perhaps encourage returns.) 

A cover Ictte^r accompanied the questionnaire. The letter, 
a sample of which is shown in Figure 4, was typed on the letter- 
head of each school and signed by the school principal. Copies 
of the letters were then printed in sufficient quantity for the 
number of graduates from each school. 

*Projcct TRACE, 6780 Cortona Drive, Golcta, California 93017. 
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SANTA SAReARA HIGH SCHOOt OlSTRtCT 



SAN MARCOS HIGH SCHOOL 

4750 Hollistcr Avenue — Santa Barbara, California — 93105 

(805) 967*4581 

Homci>J,Ahc ''ROYALS'* 



QVm a HARTLi ^ 

miNClPAL 



Dear Former Student: 



Under a special educational project. Project TRACE, a 
follow-up study of the graduates of 1970 is being con- 
ducted. The purpose of this study is to obtain informa^ 
tion from forine.r students that can be used to improve 
high school* instructional and guidance programs. Now 
that you have been out of school for a while, you have 
probably g:.vcn some thought to your high school prepara- 
tion and how it has helped you or could have helped you 
more* Your opinions are of interest to us. We want to 
improve the schools. To do this, we need your help. 

I have enclosed a questionnaire that deals with your high 
school and post-high school experiences. It will take 
you about 15 minutes to complete. I ask that you complete 
it at your earliest convenience, preferably before January 
10, and then return it in the enclosed envelope. Please 
do n'o^ put it off. 

Remember, your answers wi! help us improve the schools 
for future students, perhaps members of your own family. 
When we have completed our study, we will send you a 
letter describing its most important results. 

Your replies on the questionnaire will be kept entirely 
CONFIDENTIAL. We emphasize this because we want you to 
answer as honestly and completely as possible. 

I hope this letter finds you well. Best wishes and good 
luck to you in your future activities. 

Cordially, 

Gene G. Hartley ^ 
Principal 

Figure 4. Sample cover letter used in first mail-out to 
the class of 1970. 
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A pair of address labels was prepared for each graduate. 
One of the labels was affixed to the mailing envelope; the 
other, which in addition to the graduate • s name and address 
also had printed on it the graduate* s identification number, 
telephone number, high school identification code number, 
and his year of graduation, was affixed to the questionnaire. 
The labels were printed by^'comput ef^ and were based upon the 
information obtained from the student -record analysis ♦ In 
addition, a master list of all the graduates, containing all 
the information appearing on the questionnaire label and also 
the parent's name, was obtained on a computer print-out. 
This list was used to maintain a record of the respondents 
and non -respondents . 

Both the questionnaire and the cover letter were pre- 
folded by the printer for ease in preparing for mailing. The 
first mail-out to the 1,771 graduates of 1970 took place on 
18 December 1971* Two weeks afterward, a professional tele- 
phone-interviewing service began calling those graduates who 
had not yet responded to remind them to return the question- 
naire. One week later, a second questionnaire accompanied by 
a new cover letter (Figure 5) was mailed to the non-respondent 
Three weeks after that, the telephone-interviewing service 
began conducting interviews of the remaining non-respondents. 
An abbreviated form of the questionnaire (Figure 6) was used 
by the interviewers. In addition, the interviewers also re- 
quested the graduates to complete and return the questionnaire 
Another questionnaire was mailed to those graduates who re- 
quested them. In the event a graduate was interviewed and 
also completed a questionnaire, the telephone interview data 
was discarded. 

The same general procedures were used for the 1,941 
graduates of 1971. However, one week prior to the first mail- 
out (which took place on 10 April 1972), postcards (Figure 7) 
were sent to the graduates informing them of the purpose of 
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Project TRACE 
6780 Cortona Drive 
Coleta, Ca-lifornia 93017 



Dear Former Student: " ^ 

^^Several- weeks ago we sent you a copy of our high 
school follow-up questionnaire, with a letter 
re^i^ssting your cooperation in completing and re- 
turning the questionnaire to us. You may remember 
from our letter that the questionnaire is being 
sent to former students to enable us to obtain in- 
formation that can be used to impro\'e high school 
instructional and guidance programs. During the 
past week you*ve probably also received a tele- 
phone reminder requesting your cooperation. 

We have not yet received your completed question- 
naire and have enclosed another in case the first 
was lost or thrown away. The fact that we have 
telephoned and sent you a second copy ot the ques- 
tionnaire should indicate the importance of your 
opinions to us. Please take a few minutes of your 
time to complete and return it now. 

We have every intention of using the information 
obtained from this survey to improve your former 
high school, perhaps for the benefit of people 
you know or are related to. Filling out the ques- 
tionnaire will NOT be a waste of your time. But 
before we can do anything, we must have your help. 

Thank you for your cooperation. 



Sincerely, 




Charges Ah rams, Ph.D. 
Director, Project TRACE 



CA/lja 

Figure 5. Cover letter used in second mail-out. 
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Project trace telepuome interview 



] 



] 

1 



er|c 



Student's Name 

Student's Number.^ 



Interviewer's Initials^ 

Dat e^ 



2. 



What is the former student doing at the present tisie? Circle each 
item that describes his present activity* (Ask italicized major 
category first; if answer is "yes,*^ then ask sub-category.) 

in ScHoot g> .orU., »' -^J-J- 

In Armed Forces (3) . — Looking for a job 
Housewife (4) " ^'^^ worKvrtn looking for a job 
Other (describe) (0) 

If the forcer student is attending school full-tine or parc-tiae, 
what type of school is he attending? (Read out all the choices; 
after respond-ent indicates type of school, ask the name of the 
school and write in next to school type.) 

(1) Four-year college or university 

(2) Technical school 

(3) Trade school 



(4) Private business colleg^^ 

(5) Junior college 

(0) Oth|e r 



If the student is employed full-ti:ne or part-time, ask the following 
questions. If the student is unemployed^ ask the questions in regard 
to the last job he held* (Write in answers*) 



3, What busi.ness or industry are you employed in?^ 



What is your job title? ^ 

Which of the following statements best describes your job? 
the choices below and circle the answer,) 



(Read 



^In a field for which I received specific high school training* 
^In a field related to my high school training. 
,In a field unrelated to ny high school training. 



What educational program did you take in high school? (Read the 
choices and check one.) 

(1) College preparatory^ 



(2) Vocational program (which one? )_ 

(3) General progrr.r. _; 

(0) Other (describa) 



What educational program would you take now if you had it to do 
over again? (Read choices.) 

(1) Same program 

(2) Different program (If different, which of the following?) 
(1) College Preparatory^ 



(2) Vocational program (which one?)^ 

(3) General proc;ra v\ 

(0) Other (describe) 



Thank the respondent for his cooperation and also ask that the qucst'ioh' 
naive be ccwpl cted and returned as soon as possible. 

Indicate, who respondent was (1) Student (3) Brother or Sister 

(2) Farenc (4) Other (specify) 



Figure 6. Telephone interview data sheet. 
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Project TRACE 

6780 CORTONA DRIVE 
GOLETA, CALIFORNIA 93017 



De^^r Former Sfudenf : 

In aj^w days you will receive a c^csionnairts abouf your high school 

educafion and experiences. The^ cjucs+ionnairt& ts ^'^5 ^'^^ ou^by Projcc+ 

TRACE^ a speciaL rcs«2arc:h projzch be-jn^ conducfecTby iha, hioh schools 
in Sarrla barbara and Carpinferia .TKs^ purpose of -the projcc}- is "ho 
obJam \nJorrnc^iion Jrorn Jarm^r high school srtxiderrf3 -{hfSt can bz. u«3ecl 

Ho improve iheir schools » Ints wtlT cjiwi you 3 chance -b voice, your 
opinions aboul' boifi good and bad ^:^^^p^c■^s oj- your hj^h school ysars * 
The injforrnahon wtll course be Kept" s-incfty conJ^'c'^.rA^^L .We* hope- ihai* 
you ^ill par4ic'ipai^ by compi;rhno your Qus^+ionnainzj.l^you hnva any 
c^s+ions abouh Projed' "TRACE'^rabou+ -IheJ c^iesftonnbin&j^yGl -/rec "fo 
call Dr. Abrams or A\r. Simpson a+ ^806)^(58 - IO*7L 



Figure 7. Postcard mailed to the 1971 graduates prior to the 
first questionnaire. 
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the project and soliciting their cooperation. At the same 
time, local radio stations broadcast spot public-service 
announcements and local newspapers printed articles urging 
the graduates to cooperate and return the questionnaire. 

The data obtained from the questionnaires and telephone 
interviews were coded , keypunched onto computer cards, and 
tabulated. The two graduating class^es were treated sepa- 
rately. The characteristics of the respondents, in terms of 
the classification variables previously discussed, were com- 
pared v/ith ,the"^'s tat istical description of the population 
developed from the student-record analysis. In addition, the 
graduates* responses to the questionnaire items were analyzed 
in relation to several cross-tabulated classification variables. 

Phase 3 - Sample Selection and Evaluation 

Based upon the return rates and the differences in re- 
sponses to the questionnaire items, several classification 
variables were selected for evaluation with stratified sampling 
models. The sizes of the samples were also varied to assess 
the effects of that factor upon the results obtained with the 
various sampling models. Thus, various samples were drawn from 
the pool of respondents and the quantitative results obtained 
in response to various questions were compared with the corre- 
sponding results for the total population of respondents. 

These aspects of the sampling phase of the study--the 
selected classification variables, the sampling models, and 
the sample sizes--may be better appreciated when the results 
of the data collection are described. Therefore, a detailed 
description of Phase 3 will be deferred until those results 
are presented in the following section. 
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Results 



In this section, the findings and results obtained in 
each of the steps of Phas^ 2 of the study will be described. 

The Record Analysis 

The main purpose of tfie 'studervC-record analysis was to 
obtain data on each graduate that might be used effectively 
to classify him with respect to variables relevant to follow- 
up studies .""./'Vh'ese variables were also candidates for selec- 
tion as stratification variables for use in the development 
of sampling models. Another purpose was to obtain data which 
could be used to describe the populations statistically. 

Though a host of potential classification variables was 
examined, three were found which segmented the population 
with respett to response patterns and return rates. These 
were Sex, Ethnic Background, and Grade Point Average. Table 1 
shows the number and percentage of graduates in the various 
levels of each of these variables. Of the 1,771 graduates of 
1970, for example, 50.7 percent were males and 49*3 percent 
were females. The great majority of graduates, 83.6 percent, 
were Caucasian, 11.1 percent were Mexican-Americans, and the 
remaining small number of graduates were members of other 
minority groups. With respect to Grade Point Average, 16.2 
percent of the graduates had overall averages of less than 
2.0 on a 4-point scale; 51.8 percent had averages of between 
2.0 to 2.9, and 25.6 percent had averages of 3.0 or greater. 
For some 6.4 percent of the graduates, no G.P.A. data were 
available; one of the schools. La Cuesta, does not grade the 
students. Though the data described above were for the grad- 
uating class as a whole, analogous numbers and percentages 
are presented for the individual schools. Similar data were 
obtained for other classification variables listed in Table 2 
but were not presented since, as will be described later. 
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TABLE 1 

STATISTICAL DESCRIPTIONS OF THE POPULATIONS 
AND SCHOOLS BY THREE CLASSIFICATION VARIABLES 



CLASS OF 1970 



High Schools 







S R 




D P 


1 r 


l/CZl JJ 


POPUUTION SIZE 


1771 


666 


530 


384 


47 


144 


SEX (1) 














Male 


SO 7 




DC • iJ 


4Q n 


74 ^ 










47 S 


51 0 




43 ft 


ETHNIC BACKGROUfJb'^CI) 














Caucasian 


.83.6 


75.4 


90.0 


91.9 


83.0 


75.7 


Spanish Surname 


11.1 


17.0 


5.5 


6.3 


8.5 


18.1 


Black 


1.9 


5.0 








** 


Oriental 


1.3 


1.1 


1.5 


1.6 




1.4 


Other/DK 


2.1 


1.5 


3.0 


.2 


8.5 


4.8 


Grade point average f5"i 














Less than 2.0 


16.2 


22.1 


14.4 


9.6 




18.8 


2.0-2.9 


51.8 


51.5 


54.9 


52.3 




57.7 


3.0+ 


25.6 


21.5 


30.0 


33.4 




16.7 


DK/NA 






7 


A 7 


inn n 




ri A^^ HF 1071 














POPUUTION SIZE 


1941 


714 


528 


475 


53 


171 


SEX C^) 


• 












Male 


50 6 


46 2 


51 




«;Q * u 


R5 f) 


Female 


49 4 


53 8 


•to • *»/ 


4fi 1 


4*? 4 


44 4 


Ethnic background (I) 














Caucasian 


80.6 


68.8 


92.4 


89.4 


60.5 


75.4 


Spanisn Surname 


12.5 


20.0 


4.0 


6.7 


13.2 


23.4 


Black 


1.9 


4.3 




.2 


7.5 




Oriental 


.7 


1.3 


.2 


.6 




a* 


Other/DK 


4.3 


5.6 


3.4 


3.1 


18.8 


1.2 


GRADE POINT AVERAGE {-i) 














Less than 2.0 


16.5 


20.9 


14.6 


10.5 




25.8 


2.0-2.9 


49.8 


51.7 


51.7 


48.8 




55.0 


3.0»- 


27.9 


21.4 


33.1 


39.2 




15.7 


DK/NA 


5.8 


6.0 


.6 


1.5 


100.0 


3.5 
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TABLE 


2 


OTHER CLASSIFICATION VARIABLES 


CLASS RANK (PERCENTILE) • 




CITIZENSHIP 




LCRGE-THORNDIKE VERBAL TEST- SCORE-^( PERCENTILE) 


NO. OF YEARS ATTENDED SCHOOL 




NO. OF EXTRACURRICULAR ACTIVITIES " 


NO. OF SEMESTERS OF* 




Art 


Speech 


Business 


Work Experience 


Dance 


Agriculture 


Drafting . 


Construction Trades 


Drama 


Computer Methods 


Foreign Language 


Distributive Education 


Homemaking and 
Consumer Education 

Industrial Arts 

Math 

Music 

Science 

Social Studies 


Electronics 

Forestry 

Law Enforcement 


Motor Vehicle Repairs 
Nursery School Aide 
Nurse's Aide 

Service Station Attendant 



those variables were found to be substantially correlated with 
certain others and thus did not provide any additional useful 
basis for stratification. 

Data Colleotion 



Several steps were followed to obtain questionnaire re- 
sponses from the graduatcs'-two mail -out-- three weeks apart 
and finally a telephone interview three 1. 1 .ks after the second 
mail-out. The 1970 graduates had been out of high school about 
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18 months before receiving the first questionnaire; the 1971 
graduates had been out of high school about 10 months. Nei- 
ther of the graduating classes had been informed about the 
follow-up questionnaire or its purposes while they were still 
in school; therefore, the burden of motivating them to re- 
spond fell upon an unexpected written communication. Ideally, 
of course, a good deal of. in.dpctrinaticn would have taken 
place while the students were.stillJln school. It has gen- 
erally been found that prior orientation and discussion re- 
garding follov^-up questionnaires are necessary if the study 
is JO be successful. Nonetheless, good response rates were 
obtained from both classes. 

The response rates of the two classes for each of the data 
collection steps are presented in Table 3, It can be seen that 
the highest response rates, for both classes, were obtained 
with the first mail-out, each succeeding step producing fewer 
returns. A total of 78 percent of the 1970 graduates and 82 
percent of the 1971 graduates responded. 



TABLE 3 

PERCENT RETURNS OBTAINED 
WITH EACH DATA COLLECTION PROCEDURE 



RETURNS 

1970 1971 
(N=1 .771) (N^l ,941) 



First Questionnaire Mail -Out 


32% 


39% 


Second Questionnaire Mail-Out 


29% 


23% 


Telephone Interviews 


1755 


20% 


TOTAL RESPONDING 


7855 


B2% 


Undel i verabl c Questionnaires 


9% 


b% 


Failed to Respond 


13% 


111 


TOTAL 


100% 


100% 
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During the time span between graduation and the first 
mail-out^ some of the graduates had moved and either left 
no forwarding address or the forwarding order had expired. 
Questionnaires sent to those graduates were returned by the 
post office. For the class of 1970, the number of such re- 
turned questionnaires was 9 percent of the total mail-out; 
for the 1971 graduates, theu.total xas S percent. Attempts 
were made to trace these griadtiates through their classmates, 
community college registration rosters, and telephone direc- 
tories, and a.x^mall number was successfully located. Ques- 
tionnaires were eventually returned from various locations 
in the United States, Europe, Southeast Asia, and the South 
Pacific. 

In general, the percentages of 1970 and 1971 respondents 
at each level of the classification variables were very close 
to the population percentages. Table 4 compares the percent- 
ages for respondents and the total populations. Of particular 
interest is the fact that even though the percentages were not 
identical, the proportional makeups of the respondent pools 
were very similar to the makeups of the populations and well 
within sampling error tolerances. This indicates that the 
respondent pools of 1970 and 1971 were highly representative 
of their parent populations. 

The finding that the percentages of the respondents in 
the vaxious categories differed from the analogous population 
percentages is of some consequence in developing sampling 
models. Three classification variables in particular were 
noted to be critical in model development--Sex, Ethnic Back- 
ground, and Grade Point Average. As can be seen in Table 4, 
though the females made up 49.3 and 49.4 percent of the popu- 
lations in 1970 and 1971, respectively, they made up 50.5 and 
'51.1 percent of the respondent pools of 1970 and 1971, respec- 
tively. Similarly, the percentage of Caucasians in the re- 
spondent pools for both years was slightly gvcat-cr than the 
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TABLE 4 

PERCENT OF GRADUATES IN POPULATIONS AND AMOMG 
RESPONDENTS IN EACH LEVEL OF CLASSIFICATION VARIABLES 



SEX 



1970 

POPULATION RESPONDENTS 



1971 

POPULATION RESPONDENTS 



Male 
Female 

ETHNIC BACKGROUND 

Caucasian 
Spanish Surname 
Black 

Oriental v" 
Other , . " 

GRADE POINT AVERAGE 

Less than 2.0 

2.0-2.9 

3.0+ 

DK/NA 

CLASS RANK 

(Upper) 1-20 Percent 
21-40 Percent 
41-60 Percent 
61-80 Percent 
81-100 Percent 
OK/NA 

C ITIZENSHIP 

Less than 2.0 

2.0-2.9 
3.0+ 

DK/NA 

LORGE-THORNDIKE VERBAL TEST 

Upper 1-20 Percent 

21-40 Percent 

41-60 Percent 

61-80 Percent 

81-99 Percent 
DK/NA 

YEARS ATTENDED SC f lOOL 

1 
2 
3 
4 



50.7 
49.3 



49.5 
50.5 



50.6 
49.4 



48.9 
51.1 



83.6 ' 


85.7 


80.6 


82.6 


11.1 


9.6 


12.5 


11.2 


1 .9 


1.5 


1.9 


1.6 


1.3 


1.4 


.7 


.8 


2.1 


1.8 


4.3 


3.8 


16.2 


13.4 


16.5 


14.1 


51.8 


52.5 


49.8 


50.8 


25.6 


29.3 


27.9 


30.5 


6.4 


4.8 


5.8 


4.5 


20.3 


23.5 


19.5 


21.5 


18.9 


19.8 


19.7 


21 .2 


19.0 


19.7 


19.8 


19.6 


18.9 


17.9 


19.4 


18.8 


17.0 


14.8 


17.6. 


15.2 


5.9 


4.3 


4.0 


3.7 


1.2 


.8 


1.0 


.6 


22.9 


21.2 


20.5 


20.2 


71 .4 


74.6 


74.0 


75.8 


4.5 


3.4 


4.5 


3.4 


32.0 


33.9 


29.5 


31 .3 


19.5 


20.6 


17.9 


19.5 


17.6 


17.3 


17.1 


16.8 


15.2 


14.3 


14.7 


15.3 


11 .4 


9.6 


10.4 


8.9 


4.3 


4.3 


10.4 


8.2 


10.2 


7.3 


8.9 


7.1 


15.9 


15.3 


9.9 


9.2 


55.1 


57.7 


55.6 


57.9 


18.8 


19.7 


25.6 


25.8 



i 
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TABLE 4 
(Continued) 



N OF EXTRA - 
rUKRICULAR ACTIVITIES 

0 
1 
2 
3 
4 
5 
6 

7+ 

NllMBER_qF S EME STER^ 
FMBUSIiIESS 

0 

1 

2 
3 
4 
5 
6 
7 

8+ 

NUMBER OF SEMESTERS 
OF OWTING 

0 
1 
2 
3 
4 
5 
6 
7 

8+ 

NUMBER OF JEHESTE_RS 
W~tii D UTf R 1 A L .-TalS 

0 
1 
2 
3 
'4 
5 
6 
7 

8+ 



1970 

POPULATION RESPONDENTS 



1971 

POPULATION RESPONDENTS 



55.8 
19.3 
13.1 
6.9 
3.5 
1.1 
.2 
.1 



33.2 
14.9 
22.8 
8.5 
7.4 
3.2 
4.1 
1.6 
4.3 



85. 9 
3.0 
6.4 

.7 
1.8 

.3 
1.1 

.3 

.5 



71.9 
4.2 

10.0 
3.0 
4.1 
1.9 
1.8 
.8 
2.3 



52.8 
19.6 
14.3 
7.7 
'4T0 
U2 
.3 
.1 



34.2 
14.9 
22.8 
7.9 
7.0 
3.2 
3.9 
1.5 
4.6 



85.7 
2.8 
6.5 

.7 
1 .9 

.4 
1 .3 

.2 

.5 



74.2 
4.2 
9.9 
2.5 
3.6 
1 .4 
1.6 
.8 
1.8 



66.0 
15.1 
9.0 
5.3 
2.2 
1.5 
.7 
.2 



33.7 
15.7 
22.9 
7.4 
6.5 
3.9 
2.8 
2.1 
5.0 



82.7 
•3.7 
7.1 
1.5 
1.5 
1.1 
.7 
.6 
1.1 



73.1 
4.3 
9.2 
2.5 
3.9 
1.6 
2.0 
.7 
2.7 



63.8 
15.8 
9.3 
5.8 
2.5 
1.8 
.8 
.2 



33.4 
15.5 
23.8 
7.5 
6.2 
3.9 
2.7 
2.1 
4.9 



82.7 
3.3 
7.5 
1 .1 
1 .7 
1.3 
.7 
.6 
1.1 



73.9 
4.0 
9.5 
2.2 
3.9 
1.6 
1.8 
.7 
2.4 
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TABLE 4 
(Continued ) 



« 



1970 1971 

NUMBER OF SEMESTERS POPULATION RESPOMDEflTS POPULATION RESPOHDENTS 

OF SCIEHCE ... 

0 .t:0' 4.4 7.4 6.3 

1 2.5 2.2 4.9 4.4 

2 45.0 44.8 39.5 39.7 

3 9.4**''" ' "19.3 12.9 13.0 

4 19.9 ■ ' T9.8 16.8 17.1 

5 5.6 6.1 ^ 7.2 7.6 

6 7.7 8.4 5.5 5.5 

7 1 .8 1.7 2.3 2.5 
8+ 3.1 3.3 3.5 3.9 

NUMBER OF SEMESTCRS 
OF SOCIAL STUDIES 

0 .9 .8 1.3 .9 

1 .2 , .1 .4 .1 

2 .2 .1 1.1 .9 

3 .6 .5 1.0 .6 

4 .9 .6 3.3 2.3 

5 4.2 3.5 5.5 5.1 

6 10.4 9.6 27.2 28.4 

7 35.2 34.8 28.2 28.7 
8+ 47.4 50.0 32.0 33.0 

NUMBER OF SEMESTEP.S 
OF WORK EXPERIE.'iCE 

0 60.9 60.4 '64.4 63.4 

1 19.6 19.8 17.4 17.6 

2 11.3 11.8 10.9 11.4 

3 4.3 4.1 4.0 4.2 

4 2.5 2.2 2.6 2.7 

5 .8 1.0 .4 .5 

6 .3 .4 .1 0.0 

7 .2 .2 .1 .1 
8+ .1 .1 .1 .1 



percentage of Caucasians in the two populations while the 
Mexican-American and Black ethnic groups were slightly undor- 
I'epvesentcd among the respondents. With respect to Grade 
Point Average, those graduates who had averages of 3.0 or 
greater were ovcrrcprosented while graduates with averages of 
loss than 2.0 were underrcprcsented . The graduates with 
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averages between 2.0 and 2.9 were slightly overrepresented 
among the respondents. 

The differences between the population and respondent 
percentages are shown in- a different manner in Table S. The 
percent returns presented in this table are based upon the 
number of graduates in the populations who actually fell into 
the various categories. * FO'f exampl-e , only 76 percent of the 
1970 male graduates responded while-^a total of 80 percent of 
the 1970 female graduates responded. Eighty percent of the 
Caucasian? ^in^'lhe 1970 class responded but only 68 percent 
of * the MexicaA-Ameri can graduates responded. The percentage 
of Black graduates who responded in the 1970 class was even 
lower"-64 percent-^but the percentage of Oriental respondents 
was 83. While 89 percent of the 1970 graduates who had scho- 
lastic averages of 3.0 or greater responded, only 65 percent 
of those with averages of less than 2.0 responded. 

For the 1971 graduates, the same general relationships 
among percentages within a given variable were obtained though 
the absolute values of the percentages were generally higher. 

Examination of the other variables listed in Table 5 re- 
veals discrepant return rates among the several levels of each 
of them. With respect to Class Rank, for example, it is evi- 
dent that the response rates decreased virtually linearly as a 
function of the graduate's scholastic standing. Similarly, 
response rate can be seen to be positively correlated with 
Citizenship and also wi th achievement on the Lorge-Thorndike 
Verbal Test. Table 5 also reveals that graduates who attended 
the schools for the full, normal period of time (three or four 
years) tended to respond more readi ly than graduates who trans- 
ferred into the school sometime during their high school educa- 
tional period. Also, a positive relationship can be seen between 
response rate and the Number of Extracurricular Activities in 
which the graduates participated. 
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TABLE 5 

PERCENT RETURNS FROM THE 
VARIOUS SEGMENTS OF THE POPULATIONS 

" - PERCENT RETURNS 



1970 1971 



TOTAL . 78 82 
SEX 

Male 76 79 

Female 80 84 

ETHNIC BACKGROOND 

Caucasian 80 ' 84 

Spanish Surname 68 73 

Black 64 69 

Oriental 83 92 

Other 64 72 

GRADE POINT AVERAGE 

Less than 2.0 65 70 

2.0-2.9 79 83 

3.0+ 89 89 

CLASS RANK 

(Upper) 1-20 Percent 90 90 

21-40 Percent 81 89 

41-60 Percent 81 85 

61-80 Percent 74 79 

81-99 Percent 68 71 

CITIZENSHIP 

Less than 2.0 50 56 

2.0-2.9 72 80 

3.0+ 82 84 

LORGE-THORNDIKE VERBAL TEST 

(Upper) 1-20 Percent 84 87 

21-40 Percent 82 89 

41-60 Percent 77 80 

61-80 Percent 73 . 85 

81-99 Percent 66 70 
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TABLE 5 
(Continued) 



ERIC 



PERCENT RETURNS 
1970 1971 



YEARS ATTENDED SCHOOL 



1 56 66 

2 - ' - * - 75 76 

3 ' — 82 . 85 

4 83 84 



NUMBER OF EXTRA- 
CURRICULAR ACTIVITIES 



0 73 79 

1 79 85 

2 85 85 

3 .87 89 

4 89 91 

5 85 97 

6 100 100 
7+ 100 80 



Though all of the variables listed in Table 5 appear to 
differentiate among the graduates and therefore are candidates 
for stratification factors in sampling models, it is evident 
that several of them are highly interrelated. Thus, the use 
of all of them as stratification factors would be of very lim- 
ited value in obtaining additional representativeness in any 
sample of the population and little, if any, gain would be 
ach.i.'- -ed in reducing the error associated with sampling, as 
can be seen in Table 6, Class Rank, Lorge-Thorndike Verbal 
Test scores, and Citizenship grades are significantly corre- 
lated with Grade Point Average. Because of these correlations, 
and because Grade Point Average is easily obtained from the 
student record, there is no need to include the other variables 
in the sampling model. 
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TABLE 6 




PEARSON PRODUCT-MOMENT COEFFICIENTS OF 
CORRELATION (r) BETWEEN GRADE POINT AVERAGE 
AND OTHER VARIABLES FOR 1970 AND 1971 POPULATIONS 




1970 —1971 


Grade Point Average and Lorge-Thorndl ke 


.60 .54 


Grade ''oint Average and Class Rank 


.96 .89 


Grade Po\n.t Average and Citizenship 


• .55 .51 



Two other variables, years at school and number of extra- 
curricular activities, also appear to differentiate among the 
graduates. As can be seen in Table 5, differences in return 
rates existed between those graduates who were enrolled in the 
schools for normal time periods (3 and 4 years) and those who 
were enrolled for lesser periods (1 or 2 years). However, 
inclusion of that variable as a stratification factor would 
serve to increase the number of strata in the model which, as 
discussed earlier, is not desirable from a practical and ad- 
ministrative standpoint. Additional examination of that 
variable is necessary to determine its cogency and usefulness 
in the sampling model. With respect to the extracurricular 
activity variable, it was found that pertinent data were 
difficult to obtain in certain schools and the cost-effective- 
ness related to the use of this variable suggests that it 
should not be used as a stratification factor. The issue of 
cost-effectiveness will be discussed in more detail later in 
this report. 

One potential classification variable, which heretofore 
has not been discussed, is high school educational program. 
The three main levels of this variable are College Preparatory, 
General, and Vocational; these arc usually defined in terms of 
the number and typos of courses or course programs in which the 
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student participated. Earlier, it was stated that as part of 
the student-record analysis, data were obtained on the number 
of semester courses taken by students in various subject mat- 
ter areas. A partial, representative, list of these data and 
the associated respondent' data appear in Table 4. As can be 
seen, the population and respondent values are virtually 
identical. No bias is evidcfhr among'-'the data to suggest that 
graduates who hiid been enrolle^d in certain educational pro- 
grams (by virtue of courses taken) responded more readily 
than graduates^ enrolled in other programs. It was therefore 
not necessary to include high school educational program as 
a stratification variable in this study; however, the desir- 
ability of doing so in subsequent follow-up studies will be 
discussed in detail in the next section of the report. 

In summary, the results obtained indicated that the 
respondents of the classes of 1970 and 1971 were highly rep- 
resentative of their respective populations. The results 
also revealed that three particularly cogent variables--Sex , 
Ethnic Background, and Grade Point Average --were associated 
with the response rates of the graduates and should therefore 
be evaluated for use in the sampling models. Though not dis- 
cussed earlier, these variables were also noted, in some 
instances, to be related to differences among the kind of 
responses given by the graduates to items in the question- 
naire. This finding further supports the use and evaluation 
of these variables in the sampling models. 

Sampling Procedures 

It may be of benefit to the reader if, prior to present- 
ing the results obtained with the various sampling models, the 
techniques and procedures involved in implementing the models 
were described briefly. Two classes of samples were evaluated, 
random and st rat if ied-random. 
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Although several equally good methods for selecting random 
samples are available, the method used in the present study was 
chosen for convenience. The steps were: 

1« Determine the salnpld size desired. 

2. Assign a number to each member of the popu- 
lation. If there are, say, 500 members in 
the population, the— assignmejit would be from 
001 to 500. 

3. Divide the number of members (500) by the 
sample^^size (say, 100). The resulting 
valile^ ,5 in this case, is the skip number. 

4. Select a start number from a table of random 
numbers. Because the skip number is 5, the 
start number would be any number between 1 
and 5. 

5. Draw the sample. The first member drawn 
would be the one whose assigned number 
corresponded to the selected start number. 
Then, using the skip number as the sam- 
pling interval, the other members of the 
sample are drawn. If, for example, the 
selected start number was 3, the first mem- 
ber of the sample would be the individual 
assigned the number 3 . Every f i f th number 
from that point on would be drawn; thus, 
008 would be drawn, then 013, 018, and so 
on , unti 1 the sample size equaled 100. 

In selecting a strati fied-random sample, there are a few 
additional procedures. There are: 

1. * Select the stratification variables. 

2. Partition the population into the appro- 
priate strata. If, for example, the 
stratification variables selected are 

Sex and Grade Point Average and the 1 att er 
variab 1 e is subdivided into three 1 evcl s , 
each of the, popul ati on members would be 
classified in accordance with one of the 
six possibl e combinations of these vari- 
ables. A hypothetical example is shown 
in Table 7. 
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TABLE 7 

NUMBER AND PROPORTION OF POPULATION MEMBERS 
IN EACH SEX X GRADE POINT AVERAGE STRATUM 
(HYPOTHETICAL CASE) 

SEX 



CPA 


Male 


Female 




<2.0 


100 (p = .10)' 


70'~(p = .07) 


170 


2.0-2.9 


250 (p = .25) 


230 (p = .23) 


480 


3.0+ \ . — 


ISO (p = .15) 


200 (p = .20) 


350 




500 


500 


1000 



Popula 
tlon 



3. Determine the proportion of the population in 
each stratum using the formula 

Number of Members in Stratum 
Total Number of Population " Proportion 

In Table 7, the size of the hypothetical popu- 
lation is rOOO. The number of Males with 
G.P.A.'s of less than 2.0 is 100. The pro- 
portion is therefore .10; similarly, the pro- 
portion of Males with G*P.A,*s of 2.0 to 2.9 
is .25, and so forth. 



4. Determine total sample size desired. (Consid- 
erations relating^ to sample size are presented 
later. For this example, assume that a sample 
size of 200 is desired.) 

5. Determine stratum sample sizes using the formula 

Total Sample Size x Proportion of Population in 
Stratum = Stratum Sample Size 

Table S shows the stratum sizes based upon the 
proportions obtained in Table 7. The number 
of Males with G.P.A.'s of 2.0 or less to be 
included in the total sample is 20 (200 x .10); 
the number of Males with G.P.A.'s of 2.0 to 2.9 
is SO (200 X .25). The same procedure is used 
to determine the sample size for each of the 
other strata* 
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TABLE 8 

^ GRADE POINT AVERAGE 
STRATUM SAMPLE SIZE AND START AND SKIP NUMBERS 
(HYPOTHETICAL CASE) 



CPA 


Male 




Female 




<2.0 


20 (2, 


5) 


14 (3. 5) 


34 


2.0-2.9 


50 (1. 


5) 


46 (4. 5) 


96 


3.0+ \\"^' 


■30 (2. 


5) 


40 (5. 5) 


70 




•100 




100 


200 



Sampl e 
Size 



6. 



7. 



8. 



9. 



Assign a number to each member of each stratum. 
In Table 7, it was shown that there were 100 
Males with G.P.A.'s of less than 2.0. Each is 
assigned a number from 001 to 100. There were 
250 Males with G.P.A.'s of between 2.0 to 2.9. 
Each is assigned a number from 001 to 250. 
And so on for each stratum. 

Determine skip number for each stratum using 
the formula 

Number of Members in Stratum 

Number in Stratum Sample ^'^^P Number 

The values used in this formula are obtained 
from Tables 7 and 8. There are 100 Males 
with G.P.A.'s of less than 2.0 (Table 7) of 
which 20 will be used in the stratified sample 
(Table S) . The skip number is therefore 5 
(100/20). The skip numbers for each stratum 
are indicated in Table S. In all cases, the 
number is 5. 

Select a start number for each stratum using 
a table of random numbers. 



Draw the appropriate sized sample from each 
stratum using the identified start and skip 
numbers. The same stops described for draw- 
mg a random sample are used. 



\ 
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Another consideration in implementing sampling models is 
that of the mail-out requirements. Because it is unlikely 
that every individual member in a sample will respond, it be- 
comes necessary to over-^sampte , i.e.^ include an additional 
number of individuals in the sample above that of the desired 
sample size. There is further discussion of this aspect of 
sampling in the next section of this report • 

Sampling Models 

Three, classification variables--Sex, Ethnic Background, 
and Grade Point Average--were found to affect the results ob- 
tained with the questionnaire. These variables were therefore 
evaluated with several sampling models. Examination of the 
1970 data indicated that three stratif ied-random sampling 
models and a random sampling model should be evaluated. In 
addition, since size is an important factor in obtaining re- 
liable data from samples, it was decided to evaluate four 
different sample sizes within each sampling model. Further, 
it was decided to treat the two populations, the classes of 
1970 and 1971, separately and to validate .the findings of the 
sampling models obtained with the 1970 data with the findings 
obtained with 1971 data using the same models. 

Ten items from the questionnaire were selected for sam- 
pling model evaluation. They were selected on the basis of 
differences in responses made to them bv the various segments 
of the population and also on the basis of differences in the 
absolute percentages associated with the alternative responses. 
The ten items arc shown in Table 9 along with the population 
percentages associated with each item. The associated per- 
centages can be seen to vary from 5.0 to .SO. 7 for the 1970 
population and from 6.2 to 50.4 for the 1971 population. Also 
indicated arc the base numbers used in computing some of the 
percentages since some of the items were applicable to only a 
certain number of graduates, i.e., those who were in school. 
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TABLE 9 






SELECTED QUESTIONNAIRE ITEMS USED IN 
MODEL EVALUATIONS AND ASSOCIATED POPULATION 


SAMPL ING 
PERCENTAGES 




Popul ation 
1970 


Percentages 
1971 


1. In school, full -time "T. '.^T 50.4 


48.8 


2. Working, full-time 


24.2 


23.6 


3. In four-y^ar college^ 


33.8 


24.6 


4. Receiving on-the-job training'' 


15.5 


17.0 


5. In field related to high school 

training^ 14.0 


13.0 


6. College preparatory program 


43.8 


42.7 


7. Industrial arts most useful 
course 


5.0 


6.2 


8. Teachers presented relevant 
Information (agree) 


50.7 


50.4 


9. High school courses useful to 
career goals (agree) 


27.3 


24.8 


10. Society favors those with 

different outlook (disagree) 


41.9 


41.3 




1970 


1971 


^Percentage base = those in school 


= 781 


919 


''Percentage base = those working = 


729 


825 


^Percentage base = those working 

or looking = 


892 


1056 



those who were working, or those who were working or looking 
for a job. The items listed reflect both behavioral (items 
1-6) and attitudinal (items 7-10) questions posed in the 
questionnaire. 

A total of 1.3S3 of the 1970 graduates responded in the 
study and a total of 1,585 of the 1971 graduates responded. 



ERIC 
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However, data regarding all three variables of interest were 
available for only 1,304 of the 1970 graduates and 1,474 of 
the 1971 graduates. The graduates for whom data on all three 
variables were available were considered as comprising the 
populations from which samples were drawn. There was no evi- 
dence to indicate that a bias existed among those graduates 
who responded but for whom^he classification data were in- 
complete. - *^ 

The four sampling models which were evaluated were: 

1. Randam* sample. 

2. Stratif ied-random sample--Sex and Ethnic 
Background. 

3. Stratif ied-random sample--Sex and Grade 
Point Average. 

4. Stratif icd-random sample--Scx, Ethnic 
Background, and Grade Point Average. 

Each model v;as evaluated with four sample sizes: 

Population Percentages 
1970 1971 



1. 


N = 


100 


8 


7 


2. 


N = 


150 


12 


10 


3. 


N = 


200 


15 


14 


4. 


N = 


250 


19 


17 



Thus, a total of 16 samples were drawn from each of the 
two populations using the procedures described earlier. Each 
sample was drawn independently; it was therefore possible 
for any given individual to be a member of several different 
samp les . 

For each sample, the percentages associated with each of 
the 10 items shown in Table 9 were computed. The absolute 
differences between the population percentages and sample 
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percentages, for each item, were calculated and a mean error 
for all 10 items was computed. The results are shown in 
Table 10. 

The mean percent error associated with the random sample 
of size 100 for the class of 1970 was 2.88 percent. The ef- 
fect of increasing the sample size upon the error obtained 
with other randomly ' drawn Tamples ' cTan be seen by reading down 
the column headed Random. As is apparent, the amount of error 
is reduced with each successive increment in sample size. The 
mean total ^efrpr obtained with the Random samples (across sizes) 
wais 2. SO pVrcent. 



Similarly presented are the mean percent errors associated 
with each of the other sampling models as a function of sample 
size. With the exception of two minor reversals (Sex x Ethnic 
and Sex x G.P.A.; N = ISO in each case), the error is shown to 
decrease as sample size is increased. This can be clearly seen 
by examining the MEAN TOTAL ERROR column at the far right of 
the table. — 

With respect to the sampling models, it can be seen that 
there is generally a decrease in error associated with the 
stratif ied-random samples as compared to the random samples. 
However, here again some minor reversals are evident. With a 
sample size of 150, an insignificant but slightly higher error 
ai/20 of 1 percent) resulted with the Sex x Ethnic model as 
compared to the Randora model. A slightly higher, but again 
insignificant, error was obtained with the Sex x Ethnic model 
as compared to the Random model with an N 200. The errors 
associated with the other two strati fied-rardom sampling models 
were smaller for each N than were the Random model errors. 
This is clearly seen by comparing the MEAN TOTAL ERROR asso- 
ciated with each model, across the bottom of the 1970 table. 

Comparison of the three stratified sampling models indi- 
cates that the Sex x G.P.A. and Sox x Ethnic x G.P.A. models 
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TABLE 10 

ERROR OF ESTIMATE ACROSS TEN SELECTED 
QUESTIONNAIRE ITEMS BY SAMPLING MODEL AND SAMPLE SIZE* 



Sample 
Size 



MEAN 
TOTAL ERROR 



100, 
150 
200 
250 



- -1970 CLASS 
Sampling Model 



Random 


Sex X ' 
Ethn ic 


Sex -X 
GPA 


Sex X i 
Ethnic 1 
X GPA ! 


MEAN 

TOTAL 

ERROR 


..-2.88 


2.81 


2.44 


2.61 ! 


2.68 


"2.82 


2.87 


2.50 


2.15 


2.58 


2.31 


2.50 


1 .66 


2.01 


2.12 


1 .97 


1 .94 


1 .60 


1 .66 


1 .79 


2.50 


2.53 


2.05 


2.10 





I 
J 



1971 CLASS 
Sampling Model 



1 
] 





Random 


Sex X 
Ethnic 


Sex X 
GPA 


Sex X 1 
Ethnic 1 
X GPA 1 


MEAN 
.TOTAL 
ERROR 


1970/ 
1971 


100 


3.93 


3.87 


3.34 


3.60 


3.68 




3.18 


Sample 
^ii^ 200 


4.48 


2.18 


3.32 


2.94 


3.32 




2.95 


2.19 


3.77 


2.02 


2.33 


2.58 




2.35 


250 


2.75 


1 .93 


2.03 


2.37 

4 


2.27 




2.03 


MEAN 
TOTAL ERROR 


3.34 


2.93 


2.68 


2.81 




















1970/1971 


2.91 


2.73 


2.36 


2.46 







*Ccll entries are moan differences between sample and population 
percentages. 
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resulted in less error than did the Sex x Ethnic model. This 
was true for each level of sample size. Though it would be 
expected that a lesser amount of error would be associated 
with the Sex x Ethnic x G.P.A. model as compared with the Sex 
x G#P.A. model, such was the case only when N = 150; with the 
other sample sizes a slightly higher error was obtained with 
the Sex x Ethnic x G. P . A. jjiQdel . .X^cse reversals from the ex- 
pected are, however, well within the limits of sampling errors. 
It should be noted that only one sample of each type was drawn 
for evaluati,Y,e purposes. Theoretically, if"a large number of 
samples of .ea<:h sort were drawn and the distributions of the 
means of the samples compared, it would be found that the least 
amount of variability in the sample means (and consequently the 
least error) would occur with the Sex x Ethnic x G.P.A, model. 
This is shown in Figure 8 which depicts hypothetical distribu- 
tion of means from random and s trat if ied-random samples. In 
each of the four cases, the grand mean (mean of the means) has 
the same value; however, the distributions of the means around 
the grand mean differ. Approximately 68 percent of all the 
means fall between -lo- (standard error); the closer o- is to 
the grand mean along the lateral axis, the less the dispersion 
of the means around the gr^nd mean. Examination of the four 
cases reveals that the greatest dispersion occurs in Case 1, 
the random sample means ^ and the least dispersion occurs in 
Case 4, the three- variable strat if ied-random samp le means . 
The dispersion in Cases 2 and 3 are equal since in both cases 
stratification was on two variables and the assumption was 
made that the ethnic and grade point average factors were 
equally cogent. 

The results obtained with the same sampling models but 
using the 1971 data arc also shown in Table 10. As with the 
1970 data, the mean percent error is shown to decrease as 
sample size is increased. And , the stratified sampling models 
resulted in less overall error than did the random models. 
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CASE 3 

SEX X GRADE 
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CASE 4 

SEX X 

ETHtnc X gr;c£ 

POINT AV£=tASE 
STRATif ItO 
SAMPLE 




''x Mean 



Figure 8. Hypothetical distributions of sample nieans^ 
showing decreases in variability as a result of strati- 
' fication. Approximately 68 percent of the sample means 
fall between -l^^j^* 

There were, however, some exceptions to the general trend. 
With a sample si::e of 200 , the random model resulted in less 
error than did the Sex x Ethnic and Sex x Hthnic x G»Pw\. 
models* Among the stratified models, generally less error 
resulted with the Sex x G,P,A* and Sex x Ethnic x G*P*A, than 
with the Sex x Rthnic model though here again some reversals 
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are evident. And, as with the 1970 data, the Sex x G.P.A. 
model resulted in less error than did the Sex x Ethnic x 
G.P.A. model. However, the same comments made earlier with 
respect to the number of samples drawn (one versus some 
larger number) apply to the .1971 sampling model results as 
well . 

. The two separate sub-taBles appearing in the margins 
of Table 10 show the mean errors averaged across the 1970 
and 1971 data for the several sample models and sample 
sizes. The " sub-table appearing at the right shows that the 
wean percent"" error systematically decreased as sample size 
was increased. The sub-table appearing at the bottom shows 
that smaller mean error resulted with the stratified models 
than with the random models, and that the Sex x G.P.A. model 
produced the smallest error of all models tested. 
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DISCUSSION AND CONCLUSIONS 



The main objective^ of this :>tudy were to develop and 
evaluate procedures to be used in follow-up studies of high 
school graduates, to develop a sel f -admini s t erabl e question- 
naire which would provide actionable information and data 
from these graduates, to isolate pertinent stratification 
variablas , and to evaluate various sampling models with re- 
spect to their possible application to follow-up studies. 
These object i\;ps have been met. , 

Data Collection Procedures 

The data collection procedures used provided excellent 
return rates from the two populations. Seventy- eight percent 
of the 1970 graduates and 82 percent of the 1971 graduates 
responded. These return rates were obtained without benefit 
of prior orientation of the graduates regarding the follow- 
up or prior exposure to and instruction for completing the 
questionnaire. There seems little doubt that even higher 
return rates would have been obtained had those preparatory 
steps been possible. It is strongly recommended ' that they 
be taken in future follow-up studies. 

It is difficult to assess the value of the orientation 
postcard or the publicity given the project during the 1971 
data collection phase of the study but <^''ch procedures might 
not be necessary if the students had an .orientation session 
prior to graduation. No large benefit appeared to accrue 
from the use of conventional postage on the mailing and re- 
turn envelopes as opposed to bulk first-class mailing and 
business reply formats. The latter approach is a more effi- 
cient and expeditious method for large mailings. The two 
mail -outs and telephone interview data collection procedures 
also appear to be optimal methods. It is unlikely that the 
mailing of a third questionnaire to previous non-respondents 
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would have resulted in as large an increment in responses as 
did the telephone interview procedure. 

The Questionnaire 

The questionnaire 'usc'd in the field for the class of 1970 
did not appear to be difficult for the graduates to understand 
and complete. However, examination of the responses to the 
individual items did suggest that a small number of graduates 
was confused by two of the items. The instructions for these 
items were altered slightly for the questionnaire used for the 
class of , Examination of the responses from that class 

indicated that the problem had been resolved. The question- 
naire shown in the text appears to be completely suitable for 
field use. 

The Stratification Variables 

Three pertinent stratification vari ab 1 es -^-Sex , Grade Point 
Average, and Ethnic Background— were isolated on the basis of 
response rates and answer patterns to the questionnaire items. 
Sex and Grade Point Average are '^asily obtained from the stu- 
dent record. It was found that Grade Point Average should be 
stratified into three levels: le-ss than 2.0, 2.0 to 2.9, and 
3.0 and greater. Though finer stratification is possible, it 
would not appreciably enhance the representativeness of a 
selected sample. Data concerning Ethnic Background were ob- 
tained by examining photographs of the graduates and their sur- 
names* In the Santa Barbara schools, the populations v;ere com- 
posed mainly of Caucasians and Mexican-Americans. The number 
of Black and Oriental graduates was small; consequently they 
were combined with other very small ethnic groups for purposes 
of analysis. Depending upon the ethnic composition of other 
schools, it may be desirable to further or differently stratify 
the ethnic variable. 

As was mentioned earlier, high school program did not 
appear to be a pertinent stratification variable in the present 
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study. However, it is recommended that this variable be in- 
cluded in future studies. This would ensure an even more 
representative and precise sample and would allow for an in- 
depth evaluation of the program curricula. This is particu- 
larly important with respect to vocational programs because 
of the increase in their number and scope and the financial 
investments in them. It. i^.jalso recommended that the various 
courses of study in the vocational ^program be stratified to 
permit finer program evaluation. Program stratification data 
for each stuc^cnt may be readily provided by'the school coun- 
selor most, fami liar with the student or by the vocational 
ed'-ation coordinator of a school or district. Inclusion of 
program as a stratification variable would be cost-effective 
inasmuch as information concerning the accomplishments of 
vocational students must be obtained as evidence for state 
and federal funding. 

The other classification variables studied do not appear 
to be pertinent or cost-effective for inclusion as stratifica- 
ti >n factors,. They were found to be either highly correlated 
with Grade Point Average or difficult to obtain from the stu- 
dent record. The additional expense which would be incurred 
in obtaining the data and including them in stratification 
does not appear to be cost-effective. 

Sampling Models 

Four different sampling models wore evaluated and the 
results validated. The findings obtained indicate that 
stratif ied-random sampling models produce more accurate and 
precise results than does a random sampling model. It is 
therefore recommended that a s trnt i f ied-random sampling model 
be used in follow-up studies. Though in the present study 
the Sex x Grade Point Averai':e x Ethnic Background stratifica- 
tion did not provide the most accurate percentages when com- 
pared to the population values, they did not differ appreciably 
from the percentages obtained with the other strati fied-random 



46 



sampling models. It seems likely that in areas where there 
are larger proportions of other ethnic groups than were en- 
countered in this study, stratification by ethnic group could 
significantly improve sample representativeness. It must also 
be emphasized once again "that only one sample from each model 
(within a given sample size) was drawn; if a larger number of 
samples from each model wcrVMrawhT the best results in terms 
of minimum sampling error might well have been obtained with 
the three- variab le stratification model. This consideration, 
in addition *to the previously discussed ease with which the 
stratification data can be obtained from the student record, 
advocates strongly the use of a strat if ied-random sampling 
model. We recommend, in fact, a four-variable stratification 
model where the fourth variable is high school program which 
was described earlier. 

Sampling Error 

In the present study, samples were drawn from the combined 
populations of the participating schools. This was done simply 
in the interest of economy and procedural convenience. However, 
similar, or possibly better, results would have been obtained 
had separate samples been drawn from each of the school popula- 
tions. This can be shown by briefly discussing the concept of 
sampling error. 

As was seen earlier, the percentages obtained with the 
Various sampling models differed somewhat from the population 
percentages. This difference is referred to as sampling 
error. Usually, population values are not known so that sam- 
pling errors cannot be determined as was done in this study. 
But methods have been dcv^elopcd whereby the amount of error 
resulting from the use of sampling techniques can be estimated. 
Having examined the results obtained with the four sampling 
models, it may be worthwhile to consider briefly the benefits 
which accrue in terms of precision and accuracy from different 
sampling conditions and models. 
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Infinite Populations ^ When random samples are drawn 
from very large or so-called infinite populations, the sam- 
pling error can be estimated using the formula 

SE* = where 

p ~ the proportion of the sample which is in a given response 
or descriptive category*;' 

q = 1-p (the proportion of the sample not in the given cate- 
gory) ; 

N ^ number iii.-the sample. 

As an example, assume that 100 high school seniors from 
across the nation are selected at random (the population from 
which they are drawn can be considered to be infinite) and 
asked whether they plan to continue their education after 
graduation. Fifty reply "yes" and 50 reply "no," How well 
do these proportions represent the entire high school senior 
population? The value of p = .50 (those who replied "yes") 
and q - 1-.50 or .50.. Substituting in the formuU., we obtain 
as the standard error of the proportion 

^^p "/n "-v Too 

This can be interpreted as meaning that the odds are about 
2 to 1 that the proportion of seniors in the population who 
plan to continue their education after graduation is between 
.45 and ,55; about 19 to 1 that the population proportion is 
between .40 and .60; and about 99 to 1 that the population pro- 
portion is between .37 and ,63. 

The results obtained with the formula depend relatively 
more upon the size of N than of p and q because the product 
of p and q remains fairly constant (between .20 and .25) for 
a large range of val ues of |> (from .27 to .73). This moans 
that error resulting from the use of random samples from very 
large or infinite populations can be reduced most effectively 
by increasing the size of the sample. 
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Finite Populations* However, when the population of in- 
terest is of a finite size and not many times as largo as the 
sample (as in the case of the populations and samples used in 
the present study), saiDpTing error is reduced by the inclusion 
of a correction factor into the previous formula. For finite 
populations, sampling error can be estimated using the formula 



where N = the. number in the population; and^the other terms 
p V - • 

arc defined as^'before. 

As an example, assume that the survey described above was 
conducted at a single high school with a graduating population 
of 500, i.e., a finite population. As before, the sample size 
was 100 and p = .50 (those who replied "yes" to the question). 
Substituting in the formula, we obtain as the standard error 



^J '-'Vo-'"^ Cl-l^ . /0025 (.80, = .045. 

Thus, sampling from a finite population as compared to an in- 
finite population results in a reduced standard error (in this 
case, a 10 percent reduction) and inferences concerning the 
population are more accurate* 

Stratified Sampling. JVhen stratified-random sampling is 
employed, the standard error can be estimated using the formula 

SB ^v/m - £i where 
pg N N 

= weighted variance of strata proportions about the total 
^ sample proportion, p, and is computed as follows: 

where ^2' " number of members in a stratum 

p , p^, p, = proportion of the stratum which is in a 
given response or descriptive category 
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p = proportion of the entire sample which is in a 
given response or descriptive category 

N - total sample size. 

An example would perhaps best demonstrate the effects of 
the correction factor for stratif ied-random sampling. Assume 
that you are stratifying oij^.^Grade JPpint Average using just two. 
levels, less than 2.0 and greater ITian 3.0. The question 
being asked of graduates is whether they are now in college. 

Suppose ^that a sample of 100 graduates is drawn: SO (n^) 
with G.P.A.'^s of <2 and 50 (n2) with G.P.A. «s of >3. Of those 
with G.P.A.*s of <2, 10 reply that they are now in college 
(Pj-l0/50=.20) ; of those with G.P.A.»s of >3, 40 reply that 
they are now in college (p2 = 40/50=. 80) . Substituting in the 
equation for we obtain 

= ~Q [50(. 20-.50)2 + 50 ( . 80- . 50) 2] = .09, and 
substituting in the formula for the standard error we obtain 



L 
I 
I 
I 
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I 
I 
I 

CP = /(>5) (.5)" Toi - 7*25-. 09 
I Ps ^ " ^ 

Thus the standard error is reduced from .05 (obtained for a 
I ' sample drawn from an infinite population) to .04 by using a 

stratified-random sample-^a reduction of 20 percent. 

I This brief discussion of sampling error serves to point 

up, once again, the benefits of stratified sampling over ran*- 

I dom sampling* Also, the additional effect ;of sampling from 

finite populations has been shown. These benefits were real- 

I ized, of course, in the results obtained with the sampling 

models evaluated. They would similarly be realized in results 

m obtained from individual samples drawn from each school popu- 

^ lation. 



= .04 



Sample Si-^e 

It has been shown, both in the discussion of sampling error 
and in the evaluation of the sampling model, that sample size is 
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of considerable importance in obtaining precise and accurate 
results. However, in selecting a sample size to be used in 
future follow-up studies, several criteria should be con- 
sidered: the manpower. 4nd dollars allocated for the study 
(costs), the precision desired in the sample statistics, the 
size of the population, and whether sampling is to be propor- 
tionate or disproportionat^'Wi th respect to certain variables 
of interest. 

No hard or fast rules exist in selecting a sample size 
and general ly.'-some trade-offs or compromises must be made 
among the criteria mentioned. A typical trade-off is between 
statistical precision and allocated costs; most often, the 
costs dictate the precision. Keeping these considerations in 
mind, the following criteria are recommended for selecting a 
sample size; 

1. The standard error of the sainple statistics 
should be equal to or less than .04; and 

2. The sample should contain at least 20 percent 
of the population members; and 

3. The total sample size should not exceed 200 
members. 

Table 1 1 shows the recommended sample sizes based upon 
these criteria for various size populations. The standa rd 
errors were computed using the formula SE = / ^ (1 - ~) and 

V P 

setting p = .50. As was discussed in the section on sampling 
error, lower or higher values of p serve to reduce the size 
of the standard error. 

As is evident from Table 11, the recommended sample sizes 
vary from 76 (for a population of ISO graduates) to 200 (for 
a population of 1000^)-;- the percent of the populations in the 
samples varies from 51 to 20 percent. For small and moderate 
size schools (from 150 to 600 graduates), the standard error 
remains constant at .040 across the range of populations 

SI 



TABLE n 

SAMPLE SIZES AfJD ASSOCIATED STANDARD ERRORS 
FOR GRADUATING CLASS POPULATIONS OF DIFFERENT SIZES 



Population 
Size 


SampLe . 
Size 
(N) 


Pprrp n f 

(N/NJ 


O OQ it VKCL i u 

Error 


1 50-1 99 


76-88 • • 


■51-44 


.040 


200-249 . 


88-97 


44-39 


.040 


250-299 


97-103 


39-34 'l 


.040 


300-349, . 


103-109 


34-31 


.040 


M ^ A A <S A 

350-399 


109-113 


31-28 


. .040 


400-449 


113-117 


28-26 


.040 


450-499 


117-121 


26-24 


.040 


500-549 


121-124 


24-23 


.040 


550-599 


124-127 


23-21 


.040-. 039 


600-649 


127-130 


21-20 


.039 


650-699 


130-140 


20 


.039-. 038 


700-749 


140-150 


20 


.038-. 037 


750-799 


150-160 


20 


.037-. 035 


800-849 


160-170 


20 


.035-. 034 


850-899 


170-180 


20 


.034-. 033 


900-949 


180-190 


20 


.033-. 032 


950-999 


190-200 


20 


.032 


1000+ 


200 


20-<20 


.032-. 035 



although the percentage N/N^ varies; for larger schools (600 
to 1000 graduates), the standard error decreases slightly as 
the population increases although the percentages N/N are 
constant at 20 percent. The recommended sample sizes thus 
adhere to the three listed criteria. In the rare instances 
where the graduating population exceeds 1000, sample sizes 
of 200 arc recommended. Although this recommendation vio- 
lates the 20 percent criterion, the standard error remains 
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well below the .040 level; in fact, the standard error will 
never exceed .035. 

It is, of course, possible to reduce in size the samples 
drawn from the larger populations so that the standard error 
would remain a constant .040 across the entire population 
range. However, this is not recommended since reduction of 
the total sample is based up-on pro"portional reduction of the 
strata subsamples. Because the strata subsamples are rela- 
tively small to begin with, in general, reduction in their 
sizes would adversely affect the reliability (accuracy) of 
the inferences which could be made from the strata samples 
to the analogous strata in the population. Similar reasoning 
applies to the indicated larger percentage of the population 
in the samples needed by the smaller schools. 

It should also be appreciated that any beneficial effects 
of correcting for stratif ied-random sampling are not reflected 
in the standard errors listed in Table 11. As shown earlier, 
correction for stratification further increases the accuracy 
of sample statistics. 

The techniques for determining individual stratum sample 
sizes were described earlier. That description was based on 
the assumption of proportionate sampling in which the strata 
sample sizes are based upon the number of members of the popu- 
lation falling into the various strata. It is sometimes nec- 
essary or desirable to obtain data from a specific stratum of 
the population which, because of its size or other considera-- 
tions, is not sufficiently represented in proportionately 
drawn samples to permit reliable inferences to the population 
stratum. It is possible, however, to sample from such a 
population stratum in a disproportionate manner so that it Is 
overrcpresented in the total sample relative to its population 
proportion. By using this technique, specific inferences to 
the population stratum may be drawn from the stratum sample 
which otherwise might not be possible. This of course serves 
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to increase the overall sample size. If for some reason it 
is necessary to maintain a fixed sample size, then the other 
strata in the sample will be underrcpresented relative to 
their population proportion and the sample statistics perti- 
nent to those strata will lose some of the precision that 
would be obtained with proportionate sampling. 

Ovev-Sampling *. ' 

Earlier it was mentioned that not all sample members 
respond and that it is therefore necessary to over-sample to 
obtain a desiYcd number of study participants. The general 
equation which can be used to determine the degree to which 
one over-samples or, put another way, the total number of 
questionnaires to be mailed out is 

Number of Mail-Outs Required = Sample Size Desired 

^ Weighting Factor 

The weighting factor is generally determined from previous 
experience* Table 12 shows examples of the application of 
this equation for hypothetical random and s tratif icd-random 
samples* The weighting factor used for the random samples 
is the average of the overall response rates from the 1970 
and 1971 graduates. Substituting in the general equation, 
it can be seen that if the desired sample size (or the number 
of respondents desired) is 100, then questionnaires should be 
mailed to 125 individuals; if the number of respondents 
desired is 150, then 188 individuals should be mailed a ques- 
tionnaire . 

The same principles are used in determining the mail-out 
requirements for strata in the stratif ied-random sampling 
model. This is illustrated with the hypothetical Sex x Grade 
Point Average model described earlier. The desired sample 
size for each stratum in the model is indicated in the matrix 
at the left along with the return rates associated with them. 
The rates indicated arc those obtained from the 1970 graduates. 
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TABLE 12 

HAIL-OUT REQUIREMENTS FOR 
RANDOM AND STRATIFIED SAMPLES 

"RANDOM SAMPLE 



Weighting Factor = .SO 
(Mean of 1970 and 
1971 Response Rates) 



SAMPLE SIZE 
•^•gESIRED- 

100 

150 
200 
250 



NUMBER OF 
HAIL-OUTS 
REQUIRED 



125 = 

188 = 
250 = 

312 = 



(100) 
^80^ 

(llOj 
^.80^ 

(200) 
(250. 



SAMPLE STRATIFIED BY SEX AND GRADE POINT AVERAGE 





SAMPLE SIZE DESIRED 






NUMBER OF 
MAIL-OUTS REQUIRED 






SEX 






SEX 




GPA 


Male 


Femal e 




GPA 


Male 


Femal e 




<2.0 


20 (64%) 


14 (66%) 


34 


<2.0 






52 


0-2.9 


50 (79%) 


46 (79%) 


96 


2.0-2.9 


62 (-f) 


53 ( ^|) 


120 


3.0+ 


30 (87%) 


40 (91%) 


70 


3.0+ 




44 (-i^) 


78 




100 


100 


200 




127 


123 


250 
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The matrix at the right shows the number of individuals who 
would have to be contacted to obtain the desired number of 
responses in each stratum. Thus , a total of 31 males with 
grade point averages of less than 2.0 would have to be con- 
tacted to ensure the desired 20 respondents; a total of 62 
males with grade point averages of 2.0 to 2.9 would have to 
bo contacted to ensure the .^^.Q.sired^5p respondents in the 
second stratum, and so on. ■ ' 

Of course, the total number of individuals who are to be 
sent quest;i-onfl'aixes should be selected from the total popula-* 
tion (in the case of strati fied-random sampling) according to 
the sample selection techniques described earlier. It cannot 
be overemphasized that proper, unbiased, sample selection is 
the key to the successful application of sampling techniques* 
An improperly selected or inadvertently biased sample, regard- 
less of its siz will not be representative of the population 
to which infe:;jnces will be made and therefore the validity of 
the inferences will be open to question. 

Conclusions 

The findings cf this study indicate that sampling tech- 
niques can be effectively and econontica lly applied in follow- 
up studies. The most appropriate sampling model for follow-up 
studies is a four-variable strat if ied-random sample model, 
incorporating Sex, Grade Point Average, Ethnic Background, and 
High School Program as stratification factors. The procedures 
outlined and the questionnaire developed also appear to be 
well-suited for the successful implementation of sampling 
techniques in follow-up studies. 
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